1. Field of the Invention
The present invention relates to a software evaluation system and a software evaluation apparatus. Particularly, the present invention relates to a software evaluation system and a software evaluation apparatus for evaluating or debugging program components such as basic modulus and commands, in development of application software or firmware for devices.
2. Description of the Related Art
In development of software or firmware, if software or firmware is evaluated only by the programmer who has actually coded the software or firmware, objective evaluation of the software or firmware cannot be achieved.
Thus, software evaluation by a third party is required. In this case, the evaluator and the programmer evaluate the software from their respective viewpoints. The programmer tends to consume time in chucking the source codes of modules and commands created by him/herself. On the other hand, the evaluator is likely to use time in conducting application tests using the once created commands and modules based on predetermined evaluation points, to totally check the software,
A typical flow of software evaluation process is as follows.
First, the evaluator performs analysis/evaluation of the target program, using a predetermined evaluation tool. If an error occurs in a region of the target program or an error is detected when performing evaluation, the evaluator reports the existence of the error to the programmer who has programmed the error region. Based on this report, the programmer searches out the source code file which forms the cause of the error, and checks and debugs the source code file.
As a conventional software evaluation tool, for example, Unexamined Japanese Patent Application KOKAI Publication No. 2001-117794 discloses a program automatic evaluation system. This system automatizes the evaluation process. An evaluation tool of this system generates evaluation point data and expected value data based on an evaluation target source code. Then, the evaluation tool sends the evaluation point data together with commands to a target computer. The target computer generates an evaluation program based on the evaluation target source code in advance. Then, the target computer analyzes the commands sent from the evaluation tool, inputs the evaluation point data into the evaluation program, and obtains output of evaluation results. Then, the target computer returns the obtained evaluation results to the evaluation tool. The evaluation tool performs evaluation by comparing the evaluation results with the expected value data.
Unexamined Japanese Patent Application KOKAI Publication No. 2001-154889 discloses an apparatus and method of displaying a program performance graph. This program performance graph display apparatus performs its unique performance analysis process on a program, and automatically displays the result of analysis. This reduces the workload of evaluation. Further, this program performance graph display apparatus displays relationship between characteristic values helpful for evaluating program modules and segments to be evaluated by a graph of three-dimensional coordinates. As a result, contents of evaluation can be rapidly comprehended.
Various efforts have been made for automation of software evaluation processes, as described above.
However, conventional software evaluation processes have the following problems. First, a lot of time and labor are required since the conventional processes include a program evaluation process, report of the evaluation results including errors and defects to the programmer, and feed-back process by the programmer.
Second, when it comes to development of large-scaled software or firmware, there are included thousands of source code files to be processed. Therefore, man cannot easily keep track of so many files.
Third, when error information is reported from the evaluator to the programmer, the programmer has to unreliably find out the source code file which is related by the error among the many files with complexity.